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IMPROVEMENTS IN OR RELATING TO IMAGE PROCESSING 

The present invention relates to image processing and, in particular, to the 
enhancement of images to assist in their interpretation. 
5 There are many techniques for the processing of images, particularly , digitised 

images, to reduce noise and assist in their interpretation. Such techniques are particularly 
important in the field of medical imaging, where images are typically noisy and difficult 
to interpret. As an example, x-ray imaging is used as a basis for many medical 
techniques and, in particular, mammography is the examination of choice for early 

10 detection of breast cancer. One of the earliest indicators of breast cancer is the presence 
of microcalcifications, which can be identified in mammograms. In the accompanying 
drawings Figure 1 illustrates some mammogram samples showing microcalcifications. 
Figures 1 A and B show isolated calcifications, while Figures 1C and D show 
microcalcification clusters. It would be useful to have image enhancement techniques 

15 which assist radiologists or clinicians in finding microcalcifications in mammogram 
images. However, it is important that such techniques miss as few clinically important 
microcalcification clusters as possible, and also do not signal too many false positives. 

Methods of detecting automatically microcalcifications in mammograms have 
been proposed, for instance in WO-A-00/52641 and WO-A-01/69533. These techniques 

20 are based on an adapted version of the image known as the h inl representation in which 
the specific imaging parameters particular to the imaging process are removed. This v 
results, in essence, in a normalised image known as the Standard Mammogram Format — ' 
(SMF) which can be displayed as an h int surface, or with the h mi values converted into 
grey levels, in which case the image resembles a conventional mammogram. The 

25 techniques for producing the h ht representation will not be repeated here, but they are 
explained in detail in WO-A-00/52641 which is incorporated herein by reference. 
Figures 2 and 3 of the accompanying drawings illustrate respectively an original 




-2- 



ma mm ogram and the different A to or SMF representations. In Figure 2 the h itU values are 
shown as a surface whereas in Figure 3 the h int values are converted into grey levels and 
displayed as an SMF akin to a conventional mammogram. 

One technique for enhancing images is known as "diffusion". This is, in essence, 
5 a smoothing process in which the image is processed by convolving the intensity values 
in the image with a kernel for instance a Gaussian kernel. Although such a smoothing 
process can assist in enhancing images, it can also create problems. In particular, in an 
image containing an object shown against a background, smoothing or blurring of the 
object into the background is undesirable. Therefore so called "anisotropic diffusion" 
10 techniques have been proposed in which the diffusion processing occurs within objects, 
and within the background, but not across the boundaries between the two. Such 
techniques are disclosed, for instance, in "Scale-Space and Edge Detection Using 
Anisotropic Diffusion" by Perona and Malik (IEEE Transactions on Pattern Analysis 
and Machine Intelligence, volume 12, number 7, July 1990) and "Robust Anisotropic 
15 Diffusion" by Black et al. (IEEE Transactions on Image Processing, volume 7, number 
3, March 1998) which are incorporated herein by reference. In these techniques, though, 
the parameters of the diffusion process which include the number of iterations, the scale 
of the kernel and the diffusion coefficient itself, are set interactively by the user. This 
would be impractical for medical image processing in which many images of different 
20 qualities and characteristics are produced, thus requiring the setting of parameters for 
each different image. 

The first aspect of the present invention provides a method of processing images 
using an anisotropic diffusion process in which the anisotropic diffusion process is 
adaptive in dependence upon the image being processed. The process adapts itself in 
25 accordance with the characteristics of the image, eg a measure of the contrast in the 
image. This adaptation is automatic, and thus does not require the user to assess the 
image and set the diffusion process parameters for each different type of image. 
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A diffusion process parameter which is made adaptive may be the diffiision 
coefficient which is dependent upon the contrast in the image. In particular it may be 
calculated from a statistical analysis or measure of the local contrast in the image, e.g. 
based on an average value and standard deviation of the local contrast values. 
5 Thus the invention involves taking an original image, possibly processing it to 

enhance it using known techniques, such as in a mammogram to produce the Standard 
Mammogram Format, possibly performing other enhancements such as taking the 
Gaussian derivative, and then applying an anisotropic diffusion process in which at least 
one of the parameters of the diffusion process are calculated from the image 

10 characteristics. 

It is found that this technique allows the application of anisotropic diffusion 
processing to many different images, e.g. different mammograms, on an automated 
basis. In the case of mammograms it provides enhancement of the visibility in the 
processed image of microcalcifications. 
1 5 Another aspect of the invention provides a method of segmenting an object in an 

image from the background of the image by using a contrast based segmentation 
method, such as a so-called foveal segmentation algorithm, in which the segmentation 
algorithm is made adaptive by being dependent upon the characteristics of the image 
being processed, such as the contrast. 
20 Foveal segmentation is a segmentation based on the local contrast in areas of the 

image. It is based on an analysis of human brightness perception as explained in "A New 
Image Segmentation Method Based on Human Brightness Perception and Foveal 
Adaptation" by Heucke et al (IEEE Signal Processing Letters, volume 7, number 6, June 
2000). In the technique described in that paper, areas of an image are assigned to belong 
25 to either an object or the background depending on whether the local contrast is above a 
certain minimum contrast. The minimum contrast is calculated to be the minimum 
contrast perceivable by the human eye. With this aspect of the present invention, 
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however, the segmentation technique is developed so that at least one of the parameters 
of the segmentation process is calculated from the image characteristics. This allows the 
automatic segmentation of images of different characteristics without the need for the 
user interactively to set the segmentation parameters. Conveniently the minimum 
5 contrast value is defined with respect to the contrast in the image, for instance a 

statistical analysis or measure of the local contrast in the image, e.g. based on an average 
value, and standard deviation of the local contrast values. 

The two aspects of the invention may be combined together and they are 
particularly useful for processing medical x-ray images, particularly digitised 
10 mammograms. 

The invention also extends to a computer program comprising program code 
means for executing the image processing method on a suitably programmed computer 
system, to a computer readable storage medium carrying the computer program, and to 
an image processing apparatus for executing the image processing method. 
1 5 The invention will be further described by way of example with reference to the 

accompanying drawings in which:- 

Figures 1 A to D illustrate mammograms including microcalcifications; 
Figures 2A and B illustrate respectively a mammogram and its Standard 
Mammogram Format; 



Format; 

Figure 4 is a flow diagram illustrating image processing according to one 
embodiment of the present invention; 

Figure 5 A shows an example of a mammogram containing a large calcification 
25 and several artifacts and Figure 5B illustrates the result of diffusing the image of Figure 



Figures 3 A and B illustrate a mammogram and its Standard Mammogram 



5A; 



Figure 6A illustrates an SMF image containing a microcalcification, Figure 6B 
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the diffused conversion of image A, Figure 6C a 3-D plot of the SMF image of Figure 
6 A, and Figure 6D the surface of the diffused image in Figure 6B; 

Figures 7 A through D illustrate the removal of artifact from a mammogram, 
namely successively: Figure7A illustrates the original image, Figure 7B a map of the 
5 shot-noise, Figure 7C a map of curvilinear structures in the image and Figure 7D the 
image after shot-noise and curvilinear structure removal; 

Figure 8 illustrates an original image and diffiised versions of the image with 
different parameters; 

Figure 9 illustrates an original mammogram and its surface plot together with 
10 diffused versions of the mammogram and surface plot; . 

Figures 10A and B illustrate original images with calcifications, Figures 10C and 
D illustrate gradient maps for the images of Figures 10A and B and Figures 10E and F 
illustrate diffused versions of the images of Figures 10A and B; 

Figures 1 1 A through J illustrate original SMF images alongside corresponding 
15 processed images in which the microcalcifications have been detected and marked in 
accordance with an embodiment of this invention. 

Figure 4 is a flow diagram of image processing in accordance with one 
embodiment of the invention. In this embodiment the processing according to the 
20 invention is preceded by processing which has previously been proposed to enhance the 
image. 

The grey-level original image l is first blurred using a Wiener filter. This de- 
noises the image to an extent by removing radiographic mottle, which is a source of false 
positives in detecting microcalcifications. The Wiener filter is adapted to the 
25 characteristics of radiographic noise in the original image. This technique is explained in 
Yam, M. Brady, J.M. Highnam, R.P. Englis, R.: Denoising h int Surfaces: a Physics-based 
Approach, in Medical Image Computing and Computer-Assisted Intervention 1999, 
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Springer-Verlag, Berlin Heidelberg New York (1999) 227-234, incorporated herein by 
reference. 

The next step is the generation of the Standard Mammogram Format (SMF) 5 
using the technique described in WO-A-00/52641. 
5 This may be further processed by the glare removal technique disclosed in WO- 

A-00/52641 to produce the blurred, no glare, SMF 7. 

A major source of errors in detecting microcalcifications is film-screen shot 
noise, which appears primarily from small pieces of dust on the intensifying screen and 
has visual properties which are similar to those of microcalcifications. However, because 

1 0 shot noise is caused, for example, by dust on the screen rather than by structures within 
the breast, it is characterised by the absence of blur. Therefore such shot noise may be 
detected by the absence of blur, and then removed from the image. This technique is 
described in Highnam, R.P. Brady, J.M. English, R.: Detecting Film-Screen Artifacts in 
Mammography using a Model-Based Approach, in IEEE Transactions in Medical 

15 Imaging, Vol. 18 (1999) 1016-1024 which is herein incorporated by reference. Further, 
curvilinear structures in the breast have similar visual properties to microcalcifications 
when viewed in a noisy image. It is advantageous, therefore, to use one of the available 
techniques for the removal of curvilinear structures, for example based on phase 
congruency as disclosed in Yates, K. Evans, CJ. Brady, J.M.: Improving the Brake's 

20 Mammographic Mass Detection Algorithm Using Phase Congruency, in Proceedings of 
Digital Image Computing: Techniques and Applications, Melbourne (2002), which is 
herein incorporated by reference. 

This results in an enhanced SMF 9. Figure 7 illustrates this artifact removal 
process. Figure 7A illustrates the original image and Figure 7B the shot-noise map 

25 (white dots are noise). Figure 7C illustrates the curvilinear structure map, and Figure 7D 
the "clean" image after shot-noise and CLS removal. 

Next, in accordance with the invention, the clean SMF 9 is subjected to an 



ft 
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adaptive anisotropic diffusion process to produce a diffused image 1 1, and then to 
adaptive foveal segmentation to produce a map of microcalcifications 13. These 
processes are rendered adaptive by using a parameter k which is representative of the 
local contrast in the image. This parameter is derived from a gradient map 15 whose 
5 calculation will be described below. 

The parametric format of anisotropic diffusion makes it highly dependent upon 
the fine-tuning of its input parameters. There are three parameters to be considered when 
attempting to blur an image vising anisotropic diffustion: k - the contrast, / - the time or 
number of iterations and c - the standard deviation or scale. In practice, the more 

10 complex and variable the image is in a data set, the more problematic it is to choose a 
single set of values for these parameters that work well for the entire data set. Medical 
images, and certainly mammograms, are very complex images whose appearance varies 
widely across a population (at a centre, hospital, region, country or continent), which 
makes the vital requirements of generating few false positives and fewer false negatives 

15 very difficult. 

In accordance with this embodiment of the present invention the contrast 
parameter k, which is image dependent, is varied in dependency upon the characteristics 
of the image. The time parameter t is set to be constant (i.e. a constant number of 
iterations), as is the scale. 

20 In accordance with this embodiment of the invention the adaptive anisotropic 

diffusion is conducted with parameters, in particular a contrast value, derived from use 
of a Gaussian derivative filter. Firstly, the SMF 9 is processed to derive the Gaussian 
derivative of the image in accordance with equations 2 and 3 below: - 

(2) 



2cr 2 ) 



25 
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M=\K„\I)\ (3) 



where K is the Gaussian of image / and Mthe Gaussian derivative. 



Then the values of the local contrast g f are calculated for the image in accordance 
with equation 4 below: - 

5 



The local contrast is calculated in a neighbourhood of N pixels. These values g s may be 

displayed in a gradient map as shown in Figure 10 in which Figures 10A and B are 
1 0 images including respectively an isolated calcification and a microcalcification cluster, 

and Figures 10C and D are the corresponding gradient maps. It can be seen that the 

calcifications are more visible in the gradient maps. 

In this embodiment a computed contrast value k is then calculated from the 

gradient map for the image in accordance with equation 5 below. This value is set to be 
1 5 the average value of the local contrasts plus two standard deviations. This value will be 

subsequently used in the anisotropic diffusion process and also in the foveal 

segmentation process. 



Having calculated the value k an anisotropic diffusion process is applied to the 
clean SMF 9. This involves applying a diffusion tensor similar to that disclosed in. 
Weickert, J.: Anisotropic Diffusion in Image Processing. B.B. Teubner, Stuttgart (1998) 
herein incorporated by reference, but using the eigenvalues below: - 




(4) 



k = mean(g) + 2 * std(g) 



(5) 



20 
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for gradl\> 0 



1 



where / is the initial image, I a the Gaussian smoothed image and k the calculated contrast 
measure. It can be seen that where k is high, i.e. where the contrast is high, thus 
indicative of an edge, the value of the exponential term in X x is very small, thus 
5 inhibiting diffusion across the edge. 

Figure 8 illustrates an example of an original image and diffused versions of it 
using k = 5, a = 0.6 and firstly t = 2Q iterations and secondly / = 40 iterations. It can be 
seen that the microcalcification and also noise are more visible in the diffused images. 
Figure 9A and B illustrate an original mammogram in Figure 9A and its surface plot in 
1 0 Figure 9B, together with a diffused SMF of the mammogram in Figure 9C and its 

corresponding surface plot in Figure 9D. In this case the diffusion was conducted with k 
= 15, a = 0.6 and t =5 iterations. 



mammogram containing a large calcification and several artifacts. Figure 5B shows the 
15 result of diffusing the image and the smooth background and calcification can be clearly 
distinguished. In Figure 6, Figure 6A is an SMF image containing a microcalcification 
on the left side and a large spot of noise on the lower right side. Figure 6B is a diffused 



20 distinguished. However, in Figure 6D the surface of the diffused image of Figure 6B is 
shown and the microcalcification appears as a hill with smoother edges than those of the 



Figures 5 and 6 also illustrate image diffusion. Figure 5A illustrates a sample of a 



version of Figure 6A and Figure 6C is the 3-D plot of the SMF image in Figure 6A. The 
surface plot shows an extremely noisy appearance and important structures can barely be 
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very sharp-edged noise structures in the same image, while the background is smooth. 



Therefore the important structures can be distinguished more easily. 

The final step in the process illustrated in Figure 4 is the application of an 
adaptive segmentation method, such as the foveal method explained by Heucke et al. 
5 This processing is conducted upon the diffused SMF image 1 1. A set of mean values of 
the image intensities/values is computed using masks for the inner area, its 
neighbourhood and background. The histogram of the inner surface provides the mean of 
the values in the object (|i 0 ), and the histogram of the whole image gives the mean of the 
background values (ja B ). The mean of the values in the neighbourhood (jx N ) is defined as 
10 the weighted sum of intensities with a suitable scale set for the mask. Then the 
perceivable contrast is:- 



Then a minimum contrast value is computed where \x A = 0.923 ■ ji n +0.077 • \i B . Thus 
15 C min is calculated in accordance with equation 7 below:- 






with c w set to VAr/200 ■ Thus the segmentation process is based on the value of k, 
calculated from the gradient map and thus adapted to the particular image being 




-11- 



processed. Areas where C > C min are marked as microcalcifications. 

Figure 1 1 illustrates various original SMF images in Figures 1 1 A, C, E, G and I, 
with corresponding detection maps in Figures 1 IB, D, F, H and J. The marked areas are 
those areas where the contrast C is greater than minimum contrast C mirr It can be seen, 
5 therefore, that the microcalcification are clearly visible in the segmented map 13 
illustrated in Figures 1 IB, D 3 F, H and J. 

Although the invention has been described in relation to the processing of 
mammograms and in particular the processing of mammograms in the Standard 
Mammogram Format, it should be appreciated that the techniques are applicable to 
10 mammograms which are not in that format, and also to other images, medical or non- 
medical. 

The invention also extends to a computer program for executing the image 
processing method on a suitably programmed computer system, to a computer readable 
storage medium carrying the computer program, and to an image processing apparatus 
1 5 for executing the image processing method. 
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CLAIMS 



1 . A method of processing images, comprising applying an anisotropic diffusion process 
to the image, the anisotropic diffusion process being adapted in dependence upon the 

5 contrast in the image. 

2. A method according to claim 1 wherein a difiusion coefficient in the anisotropic 
diffusion process is adapted in dependence upon the contrast in the image. 

10 3. A method according to claim 2 wherein the diffusion coefficient is calculated from 
the local contrast in the image. 

4. A method according to claim 3 wherein the difiusion coefficient is calculated from an 
average value of the local contrast in the image. 

15 

5. A method according to any one of claims 1 to 4 further comprising the steps of 
deriving a Gaussian derivative of the image and applying said anisotropic diffusion 
process to the Gaussian derivative. 

20 6. A method of processing images to segment objects in the image from background 
comprising applying a foveal segmentation algorithm to the image in which areas of the 
image are assigned to an object if the local contrast is greater than a minimum contrast 
value, wherein the minimum contrast value is defined with respect to the contrast in the 
image. 

25 

7. A method according to claim 6 wherein the minimum contrast is calculated from an 
average value of the local contrast in the image. 
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8. A method according to claim 6 or 7 wherein the local contrast is calculated from a 
weighted sum of the image intensities in the object and in the image. 

9. A method according to any one of claims 6 to 8 further comprising the steps of 
5 deriving a Gaussian derivative of the image and applying said foveal segmentation 

algorithm to the Gaussian derivative. 

10. A method according to claim 4, 7 or 8 wherein the average value of the local 
contrast in the image is calculated over the whole image. 

10 

1 1 . A method according to any one of claims 1 to 5 further comprising sementing the 
processed image using the foveal segmentation method of any one of claims 6 to 10. 

12. A method according to any one of the preceding claims wherein the image is an x- 
15 ray image. 

13. A method according to any one of the preceding claims wherein the image is a 
medical image. 

20 14. A method according to any one of the preceding claims wherein the image is a 
mammogram. 

15. A method according to claim 13 further comprising the steps of identifying areas of 
the processed image as representing microcalcifications. 

25 
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ABSTRACT 

IMPROVEMENTS IN OR RELATING TO IMAGE PROCESSING 



Image processing method, particularly suitable for processing noisy images such 
as digitised mammograms. An adaptive anisotropic diffusion processing method is 
described in which the diffusion parameter is adjusted in accordance with the contrast in 
the image. An adaptive foveal segmentation method is also described in which the 
segmentation parameters are set adaptively in accordance with the contrast in the image. 
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